{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5f599640-2e9b-4345-819d-e4113dce5ada",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 安装实验需要的库\n",
    "!pip install scikit-learn numpy matplotlib"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2632225-aa15-4382-96e0-2cf89bb14381",
   "metadata": {},
   "source": [
    "## 1.什么是岭回归\n",
    "岭回归模型(Ridge Regression)是线性回归的一种改进，主要是线性回归求得的系数可能会过大。\\\n",
    "因此，岭回归在线性回归的基础上，在线性回归模型的MSE损失函数中加入二范正则项，以惩罚过大的系数。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93cf5091-2c9c-4c88-830e-8adacabebb25",
   "metadata": {},
   "source": [
    "## 2.岭回归使用例子\n",
    "数据如下:\n",
    "|序号|1|2|3|4|5|6|\n",
    "|---|---|---|---|---|---|---|\n",
    "|x1|0|1|3|2|5|2|\n",
    "|x2|2|1|3|2|5|2|\n",
    "|y|8|7|15|14|25|18|\n",
    "\n",
    "备注：以上数据的实际关系为： **y=2x1+3x2+2**\\\n",
    "下面我们建立岭模型,用变量 x1,x2 预测y\\\n",
    "用sklearn包求解岭回归模型\\\n",
    "用python的sklearn实现岭回归模型，只需调用**linear_model.Ridge()** 函数\\\n",
    "岭回归模型Ridge共两个关键参数：\\\n",
    "👉**alpha:** alpha是系数w的惩罚系数，alpha设得越大，模型的参数就越小\\\n",
    "👉**fit_intercept:** 岭回归默认是没有阈值的，fit_intercept设为True时，则模型为带阈值模型\\\n",
    "sklearn实现岭回归模型代码如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "50884d23-559e-4aa2-a2f7-bdccb9341ca8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "代码运行结果如下：\n",
      "当前alpha:1\n",
      "模型参数： [1.97256098 2.72865854]\n",
      "模型阈值： 2.746951219512196\n"
     ]
    }
   ],
   "source": [
    "from sklearn import linear_model\n",
    "import numpy as np\n",
    "\n",
    "#输入数据\n",
    "x = np.array([[0, 2], [1, 1], [2,3],[3,2],[4,5],[5,2]])\n",
    "y = np.array([8,7,15,14,25,18])\n",
    "\n",
    "#调用sklearn的线性模型包，训练数据\n",
    "ridge = linear_model.Ridge(alpha=1,fit_intercept=True)   # 模型实例化\n",
    "ridge.fit(x,y)                                           # 模型训练\n",
    "\n",
    "#输出模型系数和阈值\n",
    "print(\"代码运行结果如下：\")\n",
    "print(\"当前alpha:\"+str(ridge.alpha))\n",
    "print(\"模型参数： \"+str(ridge.coef_))\n",
    "print(\"模型阈值： \"+str(ridge.intercept_))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24f7a7bd-916f-4163-9cbe-784e44b5bc34",
   "metadata": {},
   "source": [
    "\n",
    "将权重与阈值代入岭回归模型数学表达式，可得\\\n",
    "**y=1.97x1+2.72x2+2.74**\\\n",
    "可以看到，它与真实关系 \\\n",
    "**y=2x1+3x2+2**\\\n",
    "有所偏差，这主要是我们对权重进行了惩罚"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c93e2b2f-2a91-40c7-8233-20a3e3a0d105",
   "metadata": {},
   "source": [
    "## 3.岭迹图\n",
    "**3.1 岭迹图是什么?**\\\n",
    "岭迹图指的不同的w随着alpha的取值的变化图，示例如下：\\\n",
    "![image.png](./assets/示例图.png)\n",
    "\n",
    "其中**横轴是alpha的值**，**纵轴是各个权重参数的值**\\\n",
    "从图中可以看到，变量1与变量2的权重参数w1与w2随着alpha增大而减小\\\n",
    "绘制岭迹图的代码示例如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3ad78a63-2d60-4966-bc04-d3d5cd05fc85",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn import linear_model\n",
    "#输入数据\n",
    "x = np.array([[0, 2], [1, 1], [2,3],[3,2],[4,5],[5,2]])\n",
    "y = np.array([8,7,15,14,25,18])\n",
    "# \n",
    "alpha_list = np.linspace(0,5,20)                                        # 设置要尝试的alpha\n",
    "w_arr      = np.empty((len(alpha_list),x.shape[1]))                     # 用于记录不同alpha下的权重参数\n",
    "for i in range(len(alpha_list)):                                        # 逐个alpha训练模型\n",
    "    ridge = linear_model.Ridge(alpha=alpha_list[i],fit_intercept=True)  # 模型实例化\n",
    "    ridge.fit(x,y)                                                      # 模型训练\n",
    "    w_arr[i] = ridge.coef_                                              # 记录当前的权重参数\n",
    "\n",
    "# 设置颜色\n",
    "colors = ['#1f77b4', '#ff7f0e']\n",
    "\n",
    "for i in range(w_arr.shape[1]):                                          \n",
    "    plt.scatter(alpha_list, w_arr[:,i], facecolors='none'               # 画出每个参数的变化\n",
    "     ,edgecolors=colors[i],marker='o',label='w'+str(i+1))  \n",
    "\n",
    "plt.xlabel(\"alpha\")\n",
    "plt.ylabel(\"weight value\")\n",
    "plt.title(\"Ridge Coefficients vs Alpha\")\n",
    "plt.legend()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "f66a1dad-7270-4828-a417-9639069799be",
   "metadata": {},
   "source": [
    "## 3.2 岭迹图怎么看\n",
    "岭迹图的用途主要有两个：\\\n",
    "👉1. 用于确定alpha的取值 \\\n",
    "👉2.分析变量\\\n",
    "如何根据岭迹图确定alpha\\\n",
    "根据岭迹图确定alpha，一般紧扣如下两点思想：\\\n",
    "👉(1) w不要过大   ：w过大往往不合理，所以所取的alpha要令整体w都不要偏大\\\n",
    "👉(2) alpha尽量小：在保障w不太大的前提下，尽量取更小的alpha\\\n",
    "因为alpha过大会使损失函数过于倾向正则项，而忽视了误差项"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "20393bf4-c11d-4bfd-9b5f-404ea70a3c31",
   "metadata": {},
   "source": [
    "## 源码获取地址：https://gitee.com/zhenxin001/MachineLearning-RidgeRegression\n",
    "## 扫码获取源码\n",
    "![image.png](./assets/二维码.png)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "da4dd119-ada1-46a7-9be0-cd97e99507c9",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.21"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
